Liquid State Machine Optimization

In this thesis several possibilities are investigated for improving the performance of Liquid State Machines. A Liquid State Machine is a relatively new system that is a Machine Learning system, which is capable of coping with temporal dependencies. Basic Recurrent Neural Networks often have problems with this. One reason for this is that it takes a long time to train the Recurrent Neural Network. Liquid State Machines train much faster by using a temporal reservoir to map temporal input into a static output pattern. These output patterns can be learned by a statistical learning method. In this thesis, two different subjects are addressed. The first subject is about reducing computation time on calculating the performance. Optimization algorithms for neural networks are often computationally heavy. This is because the performance of the system, here the Liquid State Machine, needs to be evaluated. The computation time can be decreased by using other methods to evaluate the performance. The other subject that is addressed here, is in the optimization of the temporal reservoir in the Liquid State Machine. Two different algorithms are used here, namely Reinforcement Learning and a Genetic Algorithm. The goal is to find out if the algorithms can improve the performance by improving the temporal reservoir and if so, if the performance is increased that much so that it is computationally beneficial to use. The experiments using a different performance measure showed that it will probably not help in improving the performance on classification of the Liquid State Machine. The other experiment using the two different algorithms showed that Reinforcement Learning can not find a better setting for the temporal reservoir to improve the performance given the settings of the experiments. But the Genetic Algorithm is able to improve the temporal reservoir and thus improve the performance, this was tested on two different datasets. The first dataset used was a movement classification task. The results showed an improvement, but comparing it to another system, namely Evolino, the Liquid State Machine is outperformed on both classification and computation time. The second dataset used is a music classification task. The results here where more in favor of the Liquid State Machine, although it is unclear if the parameter setting for Evolino is optimal.